Vim goes INTL - Translating Messages, Menus, Encodings
Message Translation The program used for internationalizing sources is the GNU 'gettext' utility. It is discussed in detail by its creators so we won't go into it here. Instead, we'll follow the GNU 'gettext' diagram to the letter so that we can visualize the process of creating our vim.po and vim.mo files using the program. Numbers in parentheses represent the steps we'll actually carry out (1 through 5): Quote (#1) 1.1 Original C Sources ---> PO mode ---> Marked C Sources ---. | .---------<--- GNU gettext Library | .--- make <---+ | | `---------<--------------------+-----------' (#2) | | | .-----<--- PACKAGE.pot <--- xgettext <---' .---<--- PO Compendium | | | ^ | | `---. | 2.1 | `---. +---> PO mode ---. | +----> tupdate -------> LANG.pox --->--------' | | .---' | | | | | `-------------<---------------. | (#3) | +--- LANG.po <--- New LANG.pox <----' | .--- LANG.gmo <--- msgfmt <---' | | (#4) (#5) | `---> install ---> /.../LANG/PACKAGE.mo ---. | +---> "Hello world!" `-------> install ---> /.../bin/PROGRAM -------' End quote STEP #1 Get the most recent of the vim sources (vim-6.1-src1.tar.gz, vim-6.1-src2.tar.gz) from http://www.vim.org/download.php # create a temporary folder - let's call it "vim_tmp" - # anywhere on your disk: mkdir /path/to/vim_tmp # copy the vim source tarball(s) in the temporary directory: cp /original/location/vim???.tar.gz /path/to/vim_tmp # change to that directory: cd /path/to/vim_tmp # and decompress them (currently there are only two source # archives): tar xvfz vim-6.1-src1.tar.gz ; tar xvfz vim-6.1-src2.tar.gz NOTE: Several subdirectories are created, but only the 'src' directory contains really translatable strings. 1.1 We bypass this step since all candidate strings for translation in C sources are already marked with 'N_()' or '_()' in vim source files. For a full discussion of how to mark strings as translatable in C source files, please refer to the GNU 'gettext' utilities manpage. We'll return later to this step as the keywords in vim source files ('N_' and '_') are needed as an argument to the 'xgettext' command. IMPORTANT: Before proceeding to make the PO file, we'll have to tag the sources, i.e. create the 'tags' file: # since the 'src' directory is the only one containing # translatable strings, and because only *.c files plus # two more fileS (globals.h, if_perl.xs) have them, # we switch to the 'src' directory and issue: etags *.ch *.xs # to make sure all files get tagged # we could just as well have issued: etags src/*/*.* STEP #2 The command-line options we'll use are fully described in the gettext manpage. NOTE: (i) We'll add the 'join' option in the second instance of the command line so that the strings for the second keyword ('_') be extracted and appended to the same vim.po file; otherwise a second file vim.po would have to be created. (ii) We use INPUTFILE=*.ch because we know where the strings are to be exctracted from; else we must use INPUTFILE=* to exctract from all files. # We are ready to issue the 'xgettext' command - once for each # keyword, and only for the files we know they contain # # translatable strings (i.e. *.c, globals.h and if_perl.xs): xgettext -a -d vim -k N_ -s *.c *.h *.xs xgettext -a -j -d vim -k _ -s *.c *.h *.xs # It can also be invoked in full: xgettext --extract-all --default-domain=vim \ --keyword=N_ --sort-output *.c *.h *.xs xgettext --extract-all --join-existing --default-domain=vim \ --keyword=_ --sort-output *.c *.h *.xs CAUTION: Don't be surprised when opening a vim.po file in a text editor. It looks something like this: ... #: ex_cmds.c:4421 msgid "E149: Sorry, no help for %s" msgstr "" #: globals.h:1053 msgid "E14: Invalid address" msgstr "" ... Now before proceeding to get the binary text we're after (vim.mo), we'll have to translate _all_and_each_messages_ listed in the vim.po file. The translator must insert the translated sting as the value of the 'msgstr' variable (between quotation marks). To stay with the previous quotation, the Italian translation of the excerpt above is: ... #: ex_cmds.c:4421 msgid "E149: Sorry, no help for %s" msgstr "E149: Spiacente, nessun aiuto per %s" #: globals.h:1053 msgid "E14: Invalid address" msgstr "E14: Indirizzo invalido" ... 2.1 We currently have no use for the 'tupdate' command since there's no vim.po file yet; however, this program comes handy when we already have a *.po file and must update it based on a new release of sources. The command is simple enough. After unpacking the new tarballs to our "vim_tmp" directory (step 1), we issue: # change to tmp directory cd /path/to/vim_tmp # rename "vim.po" to "OLD_vim.po" assuming it's placed here: mv vim.po OLD_vim.po # update OLD_vim.po: tupdate NEW_vim.po OLD_vim.po # rename "NEW_vim.po" back to its usable filename: mv NEW_vim.po vim.po In case we don't want to go through the whole process of creating a vim.po file after a new vim-###-src#.tar.gz release, this program will take care of recreating our updated vim.po file (NEW_vim.po) from OLD_vim.po, including the old translations which will be taken over to the newly created file as long as they still match. However, it is recommended that, after a new vim_src### release, we actually start over. Just in case! STEP #3 Next step is simplicity itself--assuming all messages have already been translated in vim.po. We'll only use one option out of several that the 'msgfmt' program supports; they are fully described in the 'msgfmt' manpage. For hints on internationalizing Vim see the previous VimTip (VimTip#). # change to tmp directory where our vim.po is placed: cd /path/to/vim_tmp # create vim.mo from vim.po: msgfmt -o vim.mo vim.po # or in full-text: msgfmt --output-file=vim.mo vim.po STEP #4 Installing the vim.mo file requires that a directory in $VIMRUNTIME$ (i.e. currently /path/to/vim61) exists or is created express for the language you'll be supporting. Let's suppose the language we support is Greek (abbreviated as 'gr' or 'el'). We do: # create directory named 'el' in $VIMRUNTIME$/path/to/lang # with a standard subdirectory 'LC_MESSAGES': mkdir /path/to/vim61/lang/el mkdir /path/to/vim61/lang/el/LC_MESSAGES # copy our vim.mo file for Greek language supported: cp /path/to/vim_tmp/vim.mo /path/to/vim61/lang/el/LC_MESSAGES STEP #5 Vim detects the system's local language settings when starting and--if supported--loads it automatically; else you'll have to change default message language from within Vim using: " language {name} e.g.: lan el " now let's test if it's working by issuing an erroneous vim command--like: :HELP " and you'll get the message: Δεν είναι εντολή κειμενογράφου: HELP " that's all Greek to me: HELP Not an editor command: HELP Summary *Download and unpack the vim sources in a temporary directory. *Use GNU 'gettext' to get your template (untranslated) file. *Translate all entries in the template in your language. *Convert the translated PO file into a MO file using 'msgfmt'. *Place MO file in /vim##/lang/your_language/LC_MESSAGES folder. *Set the 'language', 'messages', 'menu' options in Vim. *Test Vim and use it with your newly installed language. Menu Translation BASICS For Latin-based languages, menu translation is fairly painless. Open a 'menu_xx_xx.latin1.vim' and use it as a template to create the menu after translating the relevant strings into the Latin-based language you intend to support. You only have to pay attention that unique letter identifiers in a given submenu don't repeat (e.g. &Open binds the letter 'O' uniquely so that the keyboard responds to Alt-O, and cannot be repeated in the "File" menu. For Latin-based languages, at least one encoding must be defined as the default encoding for a given system; Latin1 is used on all OS'. If your candidate language is non-Latin but Latin-based, --as a rough rule--do menu translations in at least one of the following encodings: *utf-8 *iso-####-# *an MS-DOS/wINDOWS code page Consult old MS-DOS (v.5 or 6.2) online help for codepage and/or country setting details. HOW-TO To create the menu file, we use one or several word processors that supports one or several of the encodings for the language we intend to support. If we can find the above encodings in the SaveAs option of our word processor(s), it's already sufficient -- depending on the number of OS' you wish to support. We can use, for example, an existing menu_xx_xx.latin1.vim template and translate the translatable strings pretty much like the gettext program prepares them by distinguishing between 'msgid "ABC"' and 'msgstr "XYZ"'. Instead we'll do it manually. Save only the translated strings of the supported language (i.e. 'msgstr "XYZ"' in our example) into a separate file using the Save As option of a word processor (e.g. MS-Word). Now we have, for example, a file containing the translated stings, say in iso-####-x with filename 'msgstr_iso-####-x'. Here's the Hungarian menu for the iso_8859-2 specification (lang/menu_hu_hu.iso_8859-2.vim): msgid "ABC" msgstr "XYZ" Quote scriptencoding iso-8859-2 " Help menu menutrans &Help &Sϊgσ menutrans &Overview Α&ttekintιs menutrans &How-to\ links &HOGYAN\ linkek menutrans &User\ Manual &Kιzikφnyv" ... End quote We would create two plain text files, one with filename 'msgid_iso-####-y', containing the left-most part of menu_hu_hu.iso_8859-2.vim: ... menutrans &Help menutrans &Overview menutrans &How-to\ links menutrans &User\ Manual ... and one with filename 'msgstr_iso-####-z' we got with the SaveAs option of our word processor, containing the right-side part of menu_hu_hu.iso_8859-2.vim: ... &Sϊgσ Α&ttekintιs &HOGYAN\ linkek &Kιzikφnyv" ... CAUTION: Do not use a spreadsheet to do the job as these apps add/remove bytes to encodings and reset counters when saved and distort them. Only use your wordprocessor's SaveAs option and make sure it's a good one at that. Then we open both files in a simple text editor (Windows Notepad or vim) and paste line by line the left-side-file to the right-side-file so that the we get the joined file. Now them as a simple text. This is, say, our save menu_aa_bb.iso_8859-cc.vim CAUTION: Do not use a legacy word processessing program for this last Save-As-Text-Only file operation. Do not use the word processor that helped with the encoding; else you might get the wrong bytes in encodings when the left-most part of the file is saved along with the rest. We can repeat the same (painstaking, yet menus are short) process with utf-8 encoding, MS-Windows codepage, and any other encoding we need to support. Encodings To create the language encodings files for Vim in any language, we first jot down the full specification of the encodings in the most common OS' for the language we intend to support. Next we create a corresponding menu_xx_yy.ABC_mn-zz.vim file for each of these supported encodings (making sure they exist!). We may use only one or several menu translations (we suggested three basic ones above, sect. 2); one of these -- the one with the translated strings -- can be used as the default encoding to reference other ones. We reference the encodings we cannot create with a word processor to the default encoding using any menu_xx_yy.ABC_mn-zz.vim as a template. Here's the Spanish menu for the MS-Windows codepage 850 specification (lang/menu_spanish_spain.850.vim): Quote source :p:h/menu_es_es.iso_8859-1.vim End quote As you can see, it contains only one line and sources this spec here from another one (i.e. es_es.iso_8859-1 in this case). See Vim goes Greek for an example. Vangelis Eustratius vangelise at lynxx dot org Comments